Modular chassis arrangement with separately cooled logic and logic power modules

ABSTRACT

A modular packet network device has a chassis in which multiple logic cards mate to the front side of an electrical signaling backplane. Logic power for the logic cards is supplied from a group of power converter cards that convert primary power to the logic voltages required by the logic cards. The power converter cards lie in a separate cooling path behind the backplane. Advantages achieved in at least some of the embodiments include removing primary power planes from the signaling backplane or portion of the backplane, providing redundant, upgradeable power modules whose individual failure does not cause logic card failure, and providing cool air to power converter circuits that would be subject to only heated air if located on the logic cards. Other embodiments are also described and claimed.

BACKGROUND

1. Field of the Invention

The present disclosure relates generally to modular packet switch/routerdesign and operation, and more particularly to internal powerdistribution for such devices.

2. Description of Related Art

Many packet switches/routers (hereinafter “switches”) are designed in amodular fashion, with exchangeable cards and modules. The heart of sucha system is a backplane with signaling connectors for each availablecard slot, e.g., the exemplary backplane 100 of FIG. 1. Backplane 100contains internal interconnections that route signals and primary powerto a large number of external connector blocks. Two signal connectorblocks RPSC0, RPSC1 provide connections for switch management cards.Fourteen signal connector blocks (LCSC1, LCSC14 are exemplary) provideconnections for line cards, which provide external ports for receivingand transmitting the packets switched through the switch. Nine signalconnector blocks (SFSC1, SFSC9 are exemplary) provide connections forswitch fabric cards, which actually pass packets between the line cards.

In addition to supporting signaling between the cards, backplane 100distributes primary power to each attached card from two redundant banksof power supplies (not shown). One power supply bank supplies 48 V “A”power to a connection point 110A, and the other power supply banksupplies 48 V “B” power to a connection point 110B. Power is transferredthrough the backplane to “A” and “B” power connector blocks (LCPC1A andLCPC1B, adjacent signal connector block LCSC1, are exemplary) locatedadjacent the signal connector blocks.

FIG. 2 shows backplane 100, partially populated with three switch fabriccards SF5, SF6, and SF7, and two line cards LC11, LC13, as a system 200.Each card uses DC/DC power converters to convert the 48 V distributionpower received through the backplane to lower voltages (e.g., 3.3 V downto 1.0 V) required by the various logic and memory devices present oneach card. For instance, switch fabric card SF7 is depicted with twoDC/DC power converters PC7-1, PC7-2 that supply power at two differentlow voltages to the logic and memory devices present on that card. Linecard LC13 (which actually is a combination of two cards, logic boardLB13 and port interface module PIM13) is depicted with four DC/DC powerconverters PC13-1, PC13-2, PC13-3, PC13-4 that supply power at fourdifferent logic voltages to the logic and memory devices present on thelogic board and the ports (e.g., port device P13-1) present on the portinterface module.

FIGS. 3 and 4 show the general direction of power flow in the FIG. 2system. Referring first to FIG. 3, backplane 100 is shown without anycards inserted. Two four-ounce copper power planes (not visible) withinbackplane 100 connect 48 V A primary power from connection point 110A toeach “A” power connector block (connector block LCPC1A is exemplary).Two other four-ounce copper power planes (not visible) within backplane100 connect 48 V B primary power from connection point 110B to each “B”power connector block (connector block LCPC1B is exemplary). Primarypower flow direction within the backplane is indicated generally by thegroups of overlaid heavy arrows 48VA and 48VB.

FIG. 4 shows backplane 100 in side view, with a line card LC13 and aswitch fabric card SF7 visible. 48V power distributed by the backplanereaches connector blocks SFPC7A and SFPC7B, where the power istransferred to internal power planes (not visible) within switch fabriccard SF7. These power planes deliver primary power to DC/DC powerconverters PC7-1 and PC7-2, which convert the primary power to two logicvoltages LV. Power at logic voltages LV is then distributed on otherinternal power planes (also not visible) within switch fabric card SF7to logic and memory located on SF7. 48V power distributed by thebackplane also reaches connector blocks LCPC13A and LCPC13B, where thepower is transferred to internal power planes (not visible) within logicboard LC13. These power planes deliver primary power to DC/DC powerconverters PC13-1 to PC13-4, which convert the primary power to fourlogic voltages LV. Power at logic voltages LV is then distributed onother internal power planes (also not visible) within logic board LB13and port interface module PIM13.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention can be best understood by reading thespecification with reference to the following Figures, in which:

FIG. 1 shows a prior art backplane;

FIG. 2 shows the FIG. 1 backplane, partially populated with prior artline and switch fabric cards;

FIG. 3 shows the FIG. 1 backplane, indicating primary power flow;

FIG. 4 depicts the FIG. 1 backplane and a connected switch fabric cardand connected line card in side view, indicating primary and logic powerflow;

FIG. 5 illustrates a three-piece backplane system according to oneembodiment;

FIG. 6 illustrates the FIG. 5 backplane system, partially populated withline cards, switch fabric cards, and power conversion cards according toan embodiment;

FIG. 7 depicts the FIG. 5 three-piece backplane system, a connectedswitch fabric card, a connected line card, and a connected powerconversion card in side view, indicating primary and logic power flow;

FIG. 8 depicts an embodiment similar to FIG. 7, but with logic powerflow and signaling flow contained in the same backplane;

FIG. 9 depicts an embodiment similar to FIG. 7, but with primary powerflow and signaling flow contained in the same backplane;

FIG. 10 depicts an embodiment similar to FIG. 7, but with thethree-piece backplane system staggered vertically, allowing overlappingpower delivery and signaling regions;

FIG. 11 depicts an embodiment similar to FIG. 9, but with a line cardusing an under-card bus to move logic power from the power conversioncard to a more central location on the line card;

FIG. 12 depicts an embodiment similar to FIG. 7, but with primary powerflow received via bus bars;

FIG. 13 depicts an embodiment similar to FIG. 12, but with powerconversion cards also supplying logic power to the switch fabric cards;

FIG. 14 shows, in top view, a fully populated backplane system accordingto an embodiment with fourteen line cards, two management cards, andsixteen power conversion cards;

FIGS. 15 to 18 contain connection diagrams representing differentembodiments for supplying logic power in the FIG. 14 configuration;

FIG. 19 shows, in top view, a partially populated backplane systemaccording to an embodiment;

FIG. 20 contains a connection diagram for supplying logic power in theFIG. 19 configuration;

FIG. 21 shows, in top view, a fully populated backplane system accordingto an embodiment with fourteen line cards, two management cards, andeight power conversion cards;

FIG. 22 contains a connection diagram for supplying logic power in theFIG. 21 configuration;

FIG. 23 illustrates a rear view of a chassis configuration according toan embodiment;

FIG. 24 illustrates a side view of the chassis configuration of FIG. 23;

FIG. 25 shows, in rear and side views, a bus bar embodiment fordistributing primary power;

FIG. 26 shows another bus bar feature that can be added to the bus barembodiments;

FIGS. 27 and 28 show, in rear view, bus bar alternate embodiments;

FIG. 29 shows an exemplary circuit board stack for a logic powerconversion card according to an embodiment; and

FIG. 30 contains an exemplary circuit block diagram for a logic powerconversion card according to an embodiment.

DETAILED DESCRIPTION

The modular switch design described in FIGS. 1 to 4 delivers clean,redundant power to each connected card. As computing power and heatdissipation requirements typically increase with each new switchgeneration, however, it has now been discovered that a number ofdisadvantages inherent in scaling the prior art switch design can beaddressed using a novel power distribution design.

In the prior art backplane and line cards, four very thick copper layersare used to carry primary power from the redundant power sources. Due tothe copper thickness of these power planes, dielectric fill duringmanufacturing is difficult—complicated by the fact that the bestdielectric materials for the high-speed signaling layers are not thebest materials for filling voids in thick layers, nor necessarily thebest for optimal power plane performance. Sophisticated multi-step boardassembly procedures are used to produce both power and signal layerswith good quality. Furthermore, the presence of these thick powerdistribution layers adds as much as 30% to the board thickness andtherefore to signaling through-hole and via lengths, which undesirablyincreases stub effects (reflections, radiated noise, etc.) on the signaltraces.

The presence of the DC/DC power converters on the line cards createschallenges as well. The power converters are generally placed at theextreme top of the line cards, preserving prime routing room and coolerair flow for the critical logic components and paths on the card. Thisplacement puts the power converters last in the cooling air stream, andthus they receive air that has been heated by first passing across allthe logic components. Thus as logic power demands increase, it becomesincreasingly difficult to cool the power converters, which decreasestheir efficiency or may even cause them to shut down on overtemperature.In a negative pressure chassis, the power converters may also be closeto the air movers, creating hot spots on the fans if the powerconverters run too hot. Finally, if a power converter fails, due to heator otherwise, the entire board is brought down with it, and must becompletely disconnected from the system and all network cables, andreturned to the vendor for repair.

It has now been discovered that a different approach can at leastpartially mitigate each of these difficulties, in a given embodiment. Inan exemplary embodiment, a set of power conversion cards are placedbehind the backplane, and populated with DC/DC power converters. Primarypower flows vertically through the power conversion cards instead ofthrough the backplane, allowing the thick, dedicated power planes to beremoved from the backplane. As logic voltages for the line cards aregenerated on the power conversion cards, no primary power planes arerequired on the line cards either. The power conversion cards do notnecessarily require the same thickness power layers as the prior artbackplane, as each card carries but a fraction of the total powerdistributed by the system. A portion of the cooling air is drawn behindthe backplane to cool the power conversion cards separate from coolingof the logic cards, providing more effective cooling for the powersupplies than that afforded by placing them after all logic devices inthe same vertical cooling stream. Also, should a power conversion cardfail, it can be replaced much more easily than a line card, and it canbe upgraded without having to upgrade any line card.

In some embodiments, sharing mechanisms allow one power conversion cardto provide power to two different line cards, providing logic powerredundancy and/or efficiency. These and other achievable advantages willbecome clear during the detailed description presented below.

FIG. 5 illustrates, in perspective view, a three-circuit-board backplanesystem 500 according to one embodiment. The first circuit board insystem 500 is a signaling board 510 that contains line card, managementcard, and switch fabric card signaling connector blocks, e.g., arrangedsimilar to those in backplane 100. Unlike backplane 100, however,signaling board 510 does not contain primary power distribution planesor power connector blocks.

Instead, a second circuit board—primary power distribution board520—distributes primary power. Primary power distribution board 520 fitsunderneath signaling board 510, presenting two rows of primary powerconnector blocks (SFPC1A and SFPC1B, for one switch fabric cardposition, are typical) for connection of switch fabric card primarypower. Further, the back side of primary power distribution board 520contains two additional rows of primary power connector blocks (PPPC16Aand PPPC16B, for one power conversion card position, are typical) forconnection of power conversion card primary power. 48 V “A” power issupplied to a connection point 522A on primary power distribution board520, and 48 V “B” power is supplied to a connection point 522B onprimary power distribution board 520. Four internal power planes inboard 520 route power and return current from the connection points tothe respective “A” and “B” power connector blocks on both sides of board520.

A third circuit board—logic power distribution board 530—routes logicpower as produced by power conversion cards mounted behind the backplaneto logic cards mounted in front of the backplane. Logic powerdistribution board 530 fits above signaling board 510, presenting tworows of logic power connector blocks (LCLP1A and LCLP1B, for one linecard position, are typical) for connection of line card logic power.Further, the back side of logic power distribution board 530 containstwo additional rows of logic power connector blocks (PPLC16A andPPLC16B, for one power conversion card position, are typical) forconnection of power conversion card-generated logic power. Variousembodiments will be presented below, after the presentation of furtherphysical backplane system embodiments, for routing logic power betweenthe power conversion card logic power connector blocks and the line cardlogic power connector blocks to achieve different operationaladvantages.

FIG. 6 illustrates a system 600 including a partially-populatedbackplane system including signaling board 510, primary powerdistribution board 520, logic power distribution board 530, three switchfabric cards SF6, SF7, SF8, two line cards LC9 and LC13, and three powerconversion cards PC11, PC15, PC16. Although the switch fabric cardscould, in a given embodiment, be the same as those found in the priorart systems, their connections to the system differ in that primarypower connector blocks are provided on a different backplane systemcircuit board than signal connector blocks. Line cards LC9 and LC13differ from prior art line cards at least in that neither line cardcontains primary-to-logic-power converters or primary power distributionplanes or connectors. Instead, connectors to logic power distributionboard 530 supply power at the needed logic voltages from one or morepower conversion cards. Each power conversion card contains powerconverters (e.g., power converters PC16-1, PC16-2, PC16-3, and PC16-4 onpower conversion card PC16) to generate various logic power voltagesneeded by the line cards.

FIG. 7 shows a side view section of system 600, with the three backplanecircuit boards 510, 520, 530, switch fabric card SF8, line card LC13,and power conversion card PC16 visible. Primary power flow in the systemis represented by thick bolded arrows, and logic power flow in thesystem is represented by thinner bolded arrows. As stated above, primarypower from redundant “A” and “B” supplies flows through primary powerdistribution board 520 and out connector blocks to the switch fabriccards and power conversion cards. Thus primary power flows through powerconnector blocks SFPC8A and SFPC8B and through internal power planes onswitch fabric card SF8 to reach power converters PC8-1 and PC8-2. Powerconverters PC8-1 and PC8-2 generate logic power at voltages used by thelogic devices mounted on switch fabric card SF8, and distribute thatpower back through internal conductive layers on switch fabric card SF8in order to power the switch fabric card logic devices.

Primary power from redundant “A” and “B” supplies also flows throughprimary power distribution board 520 and out power connector blocksPPPC16A and PPPC16B on the backside of board 520 to reach powerconversion card PC16. Primary power then flows through internal layers(or patterned portions of layers) on the PC16 circuit board to reach thepower conversion circuitry on power conversion card PC16, includingpower converters PC16-1, PC16-2, PC16-3, and PC16-4. Power convertersPC16-1, PC16-2, PC16-3, and PC16-4 generate logic power at voltages usedby the logic devices mounted on line card LC13 (and possible one or moreother line cards, as will be discussed below), and distribute that powerthrough internal layers on the PC16 circuit board to power connectorblocks PPLC16A and/or PPLC16B.

Power connector blocks PPLC16A and/or PPLC16B connect through one ormore internal layers of logic power distribution circuit board 530 topower connector blocks LCLP13A and/or LCLP13B on the opposite side ofcircuit board 530. This couples logic power to internal layers of linecard LC13, which distributes the logic power to logic and/or porttransceivers on LC13.

FIG. 8 shows a side view section of an alternate system embodiment 800,with switch fabric card SF8, line card LC13, and power conversion cardPC16 visible. Primary power flow in the system is represented by thickbolded arrows, and logic power flow in the system is represented bythinner bolded arrows. In system 800, the signaling backplane and logicpower distribution backplane of the prior embodiment have been combinedinto a single signaling/logic power distribution backplane 810. Becausethe logic power distribution connector blocks (PPLC16A, PPLC16B,LCLP13A, LCLP13B) are positioned above all signaling connector blocks(LCSC13 and SFSC8 shown), the signaling layers in circuit board 810 arenot needed to route signals in the region of the logic powerdistribution connector blocks. Thus the signaling layers can beretasked, in the upper region of backplane 810, to form logic powerdistribution planes, with one or more signaling layers assigned to eachlogic voltage in an exemplary embodiment.

FIG. 9 shows a side view section of a second alternate system embodiment900, with switch fabric card SF8, line card LC13, and power conversioncard PC16 visible. Primary power flow in the system is represented bythick bolded arrows, and logic power flow in the system is representedby thinner bolded arrows. In system 900, the signaling backplane andprimary power distribution backplane of the FIG. 7 embodiment have beencombined into a single signaling/primary power distribution backplane910. Because the primary power distribution connector blocks (SFPC8A,SFPC8B, PPPC16A, PPPC16B) are positioned below all signaling connectorblocks (LCSC13 and SFSC8 shown), the signaling layers in circuit board910 are not needed to route signals in the region of the primary powerdistribution connector blocks. Thus the signaling layers can beretasked, in the lower region of backplane 910, to form logic powerdistribution planes. Due to the current distribution requirements of theprimary power region of backplane 910, multiple signaling and/or digitalground layers present in the signaling region can be ganged in theprimary power region of the backplane to transmit the primary power. Forinstance, if the signaling region contains 12 signaling layersinterspersed between 13 digital ground layers, the frontmost sixsignaling layers can be used for “A” power delivery and the digitalground layers sandwiched between those six signaling layers can be usedfor “A” power return in the primary power distribution region of thecard. The backmost six signaling layers can be used for “B” powerdelivery and the digital ground layers sandwiched between those sixsignaling layers can be used for “B” power return. The two outermostdigital ground layers can remain as shielding layers to protect thepower layers from external noise.

Although not shown in a separate drawing, the concepts of FIGS. 8 and 9can also be combined in a single backplane design, with the middlesection of the backplane providing signaling and digital ground layers,the upper section retasking those same layers to logic power delivery,and the lower section ganging those same layers to provide primary powerdelivery.

In FIG. 7, the three circuit boards that make up the backplane systemare essentially coplanar. This feature is not necessary, however, asFIG. 10 illustrates. In FIG. 10, an embodiment 1000 uses a backplanesystem that comprises a signaling board 1010, primary power distributionboard 1020, and a logic power distribution board 1030. Unlike FIG. 7,neither power distribution board is limited in size to the availablespace under and above the signaling board. Instead, the powerdistribution boards are slightly offset backwards from the signalingboard to allow them to pass behind the signaling board. This allows anincrease in size of the power distribution boards, which may assist inrouting power in some embodiments without heavy resistive losses. Theincrease in size is potentially offset by a decrease in size of thepower conversion cards, as shown in FIG. 10. Further, referring to FIG.6, the notches in the lower corners of the signaling backplane circuitboard—used to accommodate primary power connection points 522A and 522Bon primary power distribution board 1020—can be eliminated, increasingsignal routing room on the board. As the location of the powerconversion card power connector blocks PPPC16A and PPPC16B (and theentire row of similar connectors) is shifted upwards and placed behindthe signaling board 1010 on primary power distribution board 1020, theconnection points 522A and 522B can be accommodated on primary powerdistribution board 1020 below the level of signaling board 1010,essentially at the level of and to the sides of the row of connectorblocks including SFPC8A and SFPC8B.

Also shown in FIG. 10 is a noise shield 1040, which can be optionallyinserted between the electrical signaling backplane 1010 and the powerconversion cards, primary power distribution backplane 1020, and logicpower distribution backplane 1030. Noise shield 1040 can be, e.g., abulkhead in the switch chassis, a wire mesh grounded to the chassis,etc.

FIG. 11 illustrates yet another embodiment 1100, which is similar in allrespects to embodiment 900 (FIG. 9) except for the implementation of theline card LC13′, which differs from line card LC13. Whereas line cardLC13 delivers logic power received from connectors LCLP13A and LCLP13Bdirectly to internal power planes on LC13, linecard LC13′ does not.Instead, a connector LCLP13 delivers logic power to a multiconductorlogic power ribbon LPR13, located under card LC13′ but above the cardcarrier (not shown). Logic power ribbon LPR13 terminates, approximatelycentral to card LC13′, at a terminal connector LPTC13, which deliverslogic power to internal layers of card LC13′. With logic power ribbonLPR13 constructed to provide extremely low resistance, the inclusion ofthinner logic power planes and/or traces on the line card is possible.The ribbon can also be designed to tee and deliver power to multipleterminal connectors, located as desired relative to the line cardlayout, if so desired.

FIG. 12 shows a side view section of an alternate system embodiment1200, with switch fabric card SF8, line card LC13, and power conversioncard PC16 visible. Primary power flow in the system is represented bythick bolded arrows, and logic power flow in the system is representedby thinner bolded arrows. In system 1200, the logic power distributionbackplane has been replaced by two bus bar assemblies BBAA and BBAB,respectively providing primary “A” power and return channels and primary“B” power and return channels. Although specific implementation detailsfor the bus bar assemblies are provided at a later point in thisdisclosure, visible in FIG. 12 are switch fabric card power connectorsSFPC8A and SFPC8B, and power conversion card power connectors PPPC16Aand PPPC16B, which bolt to the bus bar assemblies to tap power andsupply that power, respectively, to SF8 and PC16.

FIG. 13 shows a side view section of an alternate system embodiment 1300that, like embodiment 1200, uses bus bar assemblies BBAA and BBAB.Primary power flow in the system is represented by thick bolded arrows,and logic power flow in the system is represented by thinner boldedarrows. In system 1300, only the power conversion cards (e.g., PC16)connect to bus bar assemblies BBAA and BBAB. The power conversion cardscontain additional power converters (e.g., PC16-5 and PC16-6) on powerconversion card PC16 to convert primary power to logic power required bythe switch fabric cards. Logic power passes through a lower set of logicpower connector blocks, PPSF16A and PPSF16B, to a lower section ofsignaling backplane 1310. Signaling backplane 1310 distributes the logicpower, e.g., to logic power connector blocks SFLP8A and SFLP8B on switchfabric card SF8. The logic on SF8 is therefore powered without therequirement for power converters on the switch fabric card. Powerconversion card PC16 in this embodiment provides logic power to both oneor more switch fabric cards, through backplane 1310, and one or moreline cards, through logic power distribution card 1320. If desired,backplane 1310 and logic power distribution card 1320 could be merged asexemplified in FIG. 8.

FIG. 14 shows an embodiment 1400 in top view, showing line cards L1 toL14, management cards RPM0 and RPM1, logic power conversion cards P1 toP16, and a backplane system 1410 in edge view. The arrangement ofembodiment 1400 is compatible with all previously described backplanesystem embodiments. In embodiment 1400, the number of power conversioncards is equal to the aggregate number of line cards and managementcards. Thus one possible arrangement is for each power conversion cardto provide logic power to the line or management card directly in frontof it, e.g., P1 provides power to L1, P8 provides power to RPM0, P16provides power to L14, etc.

A logic power distribution backplane circuit board (or section ofbackplane circuit board) can route logic power in other ways than asimple one-to-one power/powered card correspondence. For instance, FIG.15 shows a connection diagram 1500, with the power conversion cardsappearing as their labels spaced around an outer circle and theline/management cards appearing as labels spaced around a smallerinscribed circle. A logic power connection from a power conversion cardto a line or management card is represented by a solid line between thetwo. In FIG. 15, two power conversion cards supply power to twoline/management cards, e.g., P1 and P4 power L1 and L4. It is believedthat ground loop noise can be better avoided by not having adjacent linecards powered by the same two power conversion cards, thus this patternis continued around the circle, with L2 and L5 powered from P2 and P5,and L3 and L6 powered from P3 and P6. In a second grouping, L7 and L8receive power from P7 and P10, RPM0 and L9 receive power from P8 andP11, and RPM1 and L10 receive power from P8 and P12. In a thirdgrouping, L11 and L13 receive power from P13 and P15, and L12 and L14receive power from P14 and P16.

The “50” labels on each power connection indicate that each powerconversion card is supplying 50% power to each connected line ormanagement card. Should one or more power converters fail on one of thepower conversion cards (or a power conversion card be removed), thepaired power conversion card can attempt to supply 100% power to bothcards that it powers, so that neither line/management card is broughtdown by a power failure on a single power conversion card. For thisredundancy to be fully effective, each power conversion card should berated to deliver the full maximum power requirements of two cards. Suchan effort can also be aided by not placing two high-power-consumptioncards (e.g., two Power Over Ethernet cards) in power-paired line cardslots when possible. Different power conversion cards with differentratings can also be used, depending on the power requirements of theprotected cards.

Other load-sharing logic power backplane designs are also possible. Forinstance, FIG. 16 shows a connection diagram 1600 for a different powerload-sharing map. In this map, to the extent possible, three powerconversion cards power three line/management cards as a group, with twoof the three power conversion cards supporting each one of the poweredcards in the group. An exemplary group L1, L4, L7 is powered from thegroup P1, P4, P7, such that no two line cards in the first group receivepower from the same two power conversion cards. Thus if one powerconverter goes down, two power converters can still share the load forthe three powered cards. Each power conversion card in such a groupingneed only be designed to generate 150% of a single line card's powerrequirements in this design, as opposed to 200% in the prior embodiment.

This same load-sharing concept can be applied to group sizes greaterthan three cards. FIG. 17 shows a connection diagram 1700 for a powerload-sharing map in which the group size encompasses all sixteen powerconversion cards and all sixteen line/management cards. Each powerconversion card powers two logic cards that are three slots apart, in asingle linked pattern. In diagram 1700, the failure or removal of asingle power conversion card can be compensated by balancing a slightincrease (6.6%) in power drawn from each of the remaining 15 powerconversion cards, with each adjusting its power share to the cards itpowers accordingly.

Such a failure scenario is illustrated in the connection diagram 1800 ofFIG. 18. Connection diagram 1800 is identical to connection diagram1700, except P1 is represented as shut down by dashed connection lineswith “0” showing the amount of power supplied from P1. P1 previouslypowered L3 and L14. L3 was previously powered 50% by P4 as well, and P4must now supply 100% power to L3. P4, however, reduces the percentage ofpower it supplies to its other powered card, L6, to 6.7%. P7 compensatesby supplying 93.3% power to L6, and reduces the percentage of power itsupplies to its other powered card, RPM1, to 13.3%. P10 compensates bysupplying 86.7% power to RPM1, and reduces the percentage of power itsupplies to its other powered card, L10, to 20%. This load-shiftingbehavior cascades until P9 is reached—P9 supplies 53.3% power to bothRPM0 and L9. Further load shifting continues along the chain, until P14is reached. P14 supplies 100% of the power to L14, since P1 is no longeravailable. Thus one benefit of this design is that a single powerfailure can be compensated by distributing the power load across allremaining power conversion cards, with only a slight increase in thepower drawn from each remaining power conversion card.

Connection diagram 1800 suggests that efficient power redundancy is alsopossible with a partially-populated backplane. FIG. 19 depicts such asituation, from a top view of an embodiment 1900. Embodiment 1900comprises: backplane system 1410; two management cards RPM0 and RPM1;nine line cards L2, L3, L5, L6, L8, L10, L11, L13, and L14; and twelvepower conversion cards P1, P3, P4, P6, P7, P8, P9, P10, P11, P13, P14,and P16.

The particular arrangement of line card and power conversion cardinsertion allows N+1 power conversion cards to provide redundant powerto N line and management cards in a partially populated system, as longas the selection of line and power conversion card slots follows thepower distribution mapping of FIG. 17. FIG. 20 shows a connectiondiagram 2000, which is the same power distribution mapping of FIG. 17,except shown with “X” markings for the unused line card and powerconversion card slots of FIG. 19. By selecting sets of empty line cardslots served by the same empty power conversion card slots, theremaining slots can be served by a supporting chain of power conversioncards. The percentages shown on connection diagram 2000 illustrate thepower sharing used with all remaining power conversion cards active—eachoperates at less than 92% load, with at least two power conversion cardsproviding power to each line and management card.

With the power distribution arrangement of FIG. 20, a customer can besupplied with a chart instructing them as to how to populate a chassisto achieve best power redundancy protection. An exemplary chart ispresented below as Table 1, which assumes that both management cardslots are populated as well:

TABLE 1 Place Linecards in Place Power Conversion Cards in # ofLinecards Slots Slots 1 L5 P3, P6-P7, P9-P10 2 L5, L10 P3, P6-P7,P9-P10, P13 3 L2, L5, L10 P3, P6-P7, P9-P10, P13, P16 4 L2, L5, L10, L13P3, P6-P7, P9-P10, P13, P16 5 L2, L5-L6, L10, P3-P4, P6-P7, P9-P10, P13,P16 L13 6 L2-L3, L5-L6, P1, P3-P4, P6-P7, P9-P10, P13, LI0, L13 P16 7L2-L3, L5-L6, P1, P3-P4, P6-P7, P9-P10, P13- LI0, L13-L14 P14, P16 8L2-L3, L5-L6, P1, P3-P4, P6-P7, P9-P11, P13- L10-L11, L13- P14, P16 L149 L2-L3, L5-L6, P1, P3-P4, P6-P11, P13-P14, P16 L8, L10-L11, L13-L14 10L2-L3, L5-L8, P1, P3-P11, P13-P14, P16 L10-L11, L13- L14 11 L2-L8,L10-L11, P1-P11, P13-P14, P16 L13-L14 12 L1-L8, L10-L11, P1-P11, P13-P16L13-L14 13 L1-L8, L10-L14 P1-P16 14 L1-L14 P1-P16FIG. 20 and Table 1 are exemplary; many other power conversion cardmappings and population patterns can be developed according to theteachings herein.

Another use of the teachings herein is to supply logic power for severalline and/or management cards from a single power conversion card. FIG.21 illustrates this concept in a top view of an embodiment 2100.Embodiment 2100 includes a backplane system 2110, two management cardsRPM0 and RPM1, fourteen line cards L1 to L14, and eight power conversioncards P1 to P8. Each power conversion card supplies logic power to twoline and/or management cards. Each power conversion card is thus ratedto supply at least 200% of a line or RPM card's maximum power rating,and can contain internally redundant power converters to safeguardagainst partial power loss on the power conversion card. Each powerconversion card can also be configured to cut power preferentially toone of its two powered cards over the other one in the case of the powerconversion card not being able to meet the power requirements of bothcards.

The powered cards can be, in a given embodiment, the twologic/management cards physically closest to a power conversion card.Alternately, backplane system 2110 can contain connections thatdistribute power in other arrangements. FIG. 22 contains a connectiondiagram 2200 for one such arrangement, where each power conversion cardsupplies power to two line or management cards that are at least twoslots apart, according to the following mapping: P1→L1, L4; P2→L3, L6;P3→L5, RPM0; P4→L7, L8; P5→RPM1, L10; P6→L9, L12; P7→L11, L14; andP8→L2, L13.

When the logic cards receive power through a logic power distributioncircuit board that is separate from the signaling backplane circuitboard, different logic power distribution circuit boards can be insertedin a chassis to achieve different effects, including those describedabove, combinations of the embodiments above, and many others. Further,upgrades in power converter technology can be implemented merely byswitching power conversion cards.

FIG. 23 illustrates, in rear view, the physical arrangement of theinterior of an unpopulated switch chassis embodiment 2300. Switchchassis 2300 includes a chassis frame 2302, e.g., for mounting in anequipment rack. The backplane system includes a signaling backplane2310, a logic power distribution backplane 2320, and “A” and “B” primarypower distribution bus bars 2330A and 2330B. Bus bars 2330A and 2330Bextend downwards behind a cooling air intake/filtering zone plenum 2350to connect, respectively, to “A” and “B” primary power supplies 2340Aand 2340B. At the extreme top of chassis frame 2302, a fan tray 2360pulls cooling air through the switch chassis 2300 and exhausts theheated cooling air.

FIG. 24 illustrates switch chassis 2300 in left side view, populatedwith a switch fabric card SF1, a line card LC1, and a power conversioncard PC1. Superimposed on switch chassis 2300 are bolded arrowsrepresenting air flow through switch chassis 2300. Starting at thebottom of chassis frame 2302, primary power supply 2340A mounts in anAC/DC primary power zone. Primary power supply 2340A has its own coolingfans, which draw cooling air in the front and exhaust spent cooling airout the back of chassis frame 2302.

Immediately above the AC/DC primary power zone, a cooling airintake/filtering zone includes plenum 2350. Cooling air is drawn intoplenum 2350 through openings in the front of chassis frame 2302. Thecooling air is turned approximately vertically, e.g., with assistancefrom a series of vanes or louvers (not shown), and preferably passesthrough a filter (not shown) that inhibits the introduction of dust intothe electronics. Although bus bars 2330A and 2230B extend through plenum2350, a large area between the bus bars allows cooling air to enter theregion behind the backplane system.

Cooling air next passes vertically through the DC distribution powerzone, where switch fabric card SF1 and power conversion card PC1 connectto bus bars 2330A and 2330B. In front of the bus bars, electronics ofSF1 will likely exist and require cooling. Behind the bus bars, theremay or may not be electronics mounted on PC1. If not, cooling air willcontinue upward across the surface of PC1.

Cooling air next passes vertically on both sides on both sides ofsignaling backplane 2310, in the backplane signaling zone. In front ofsignaling backplane 2310, cooling air in this zone cools the electronicson the remainder of SF1, and cools electronics on LC1. Behind signalingbackplane 2310, power converters on PC1 receive cooling in this zone.

The logic power delivery zone roughly coincides with the location oflogic power distribution board 2320. Cooling air passing in front ofboard 2320 cools the electronics on the top portion of LC1. Cooling airpassing behind board 2320 cools any remaining power converters on PC1.

After the cooling air passes out of the electronics, it enters an upperplenum 2362. In plenum 2362, the cooling air is drawn backwards to fantray 2360. Thus the cooling air that cools the logic boards is separatedfrom the cooling air that cools the power boards until after allelectronics have been cooled.

FIG. 25 depicts, in an enlarged view, the primary power distributionsystem 2500 of FIGS. 23 and 24. Two primary power supplies, 2340A and2340B, connect respectively to two bus bar assemblies, 2330A and 2330B.Each bus bar assembly is shown in a front view (as seen from the vantagepoint of a power conversion card mounted in a chassis) and a sidecross-section view. Features of each bus bar assembly are largelysimilar, except bus bar assembly 2330A is taller, and jogs behind theend of bus bar system 2330B.

In common system assemblies, chassis ground, primary power returnconnections, and digital ground connections are tied together at eachprimary power supply. Further, chassis ground and digital ground tietogether at a single point on the backplane. Control connections betweenthe backplane and one or more power supply units may exist as well, witha digital ground reference connected between the power supply units andthe backplane to serve the control connections. Each power conversioncard also contains a plurality of logic power converters that referenceto primary power return voltage on the input side, reference to digitalground on the output side, and contain at least one chassis groundconnection as well. These distributed multiple ground connectionspresent the potential for undesirable ground loops in the system.Further, it has now been found that radiated noise from the electronicscan intrude on the primary power through the leads from the powersupplies to the backplane. Bus bar assemblies described herein have nowbeen developed that combat these problems.

Each bus bar assembly comprises a supply conductor (AS and BS) and areturn conductor (AR and BR), e.g., fabricated from (111) copper. At thepower supply ends of the conductors, each conductor contains a baresection to facilitate establishing an electrical pathway to a powersupply. The remainder of each conductor is preferably covered with adielectric, e.g., 6 mils of polypropylene in one embodiment. Thepolypropylene is removed in specific locations along the length of thebar to allow connection to the power supply connector blocks.

As the supply and return conductors of a bus bar assembly ascend throughthe chassis from their respective power supply connections, the twoconductors' positions collapse against each other such that theconductors overlie each other in close proximity (with insulation inbetween). The conductors ascend upwards together, surrounded by a shieldmaterial that is tied to chassis ground (the shield material in theFigures is only shown behind the bus bar assemblies for clarity of otherfeatures, but in fact surrounds the bus bar assembly). The conductorsseparate at one point along their upward path in the illustratedembodiment before rejoining. The separation allows ferrite chokes 2510and 2512 to be inserted respectively around BS and BR, and similarferrite chokes (unlabeled) to be inserted around AS and AR.

As each bus bar assembly rises to the level of its connectors for thepower and switch fabric cards, the bus bar assembly bends and runshorizontally. The supply and return conductors remain partiallyoverlapped over the horizontal run, although the conductors arestaggered to allow power connectors to access both conductors from eachside. Each power conversion card connector (PPPC3A and PPPC3B areexemplary) bolts through the insulating material covering each conductorto holes aligned with the respective power conversion card slot. On theopposite side of the bus bar assemblies, switch fabric card primarypower connectors (SFPC1A and SFPC1B are exemplary) bolt through theinsulating material in the opposite direction from the power conversioncard connectors.

In one embodiment, it is desirable to reduce noise in the bus barassembly over at least the frequency range 10 kHz to 100 MHz. In FIG.25, differential mode noise is combated by the proximity of and thecapacitance existing between the two conductors of each bus barassembly. The capacitance between the conductors is enhanced by thedielectric material and by the close spacing of the conductors.Differential-mode noise is also combated by the ferrite chokes on eachconductor. Both differential-mode and common-mode noise rejection areaided by several features of the bus bar assemblies, including: thegrounded shielding surrounding the conductor pair (this shielding canalso be replaced by or used with a ferrite choke surrounding bothconductors); and discontinuities in overlap that create differences inthe incremental impedance between the conductor pairs, e.g., where theconductors bend away from each other to allow the positioning of theferrite chokes. Other features, such as a widened section 2600 of thebus bars shown in FIG. 26, can also be incorporated to create impedancediscontinuities that lower noise. When considered together, theseelements provide an extremely low-noise, low-resistance DC connectionbetween the power supplies and the power conversion cards.

FIG. 27 illustrates another primary power distribution system embodiment2700 for use with eight power conversion cards instead of sixteen. Twobus bar assemblies 2730A and 2730B connect, as in the previousembodiments, to two primary power supplies 2340A and 2340B. Fouroutboard power conversion card power connectors (PPPC1A, PPPC2A, PPPC7A,and PPPC8A, for the bus bar assembly 2730A; PPPC1B, PPPC2B, PPPC7B, andPPPC8B, for the bus bar assembly 2730B) bolt to the power conversioncard side of the bus bar assemblies. The remainder of the powerconversion card power connectors are paired with switch fabric cardpower connectors in a common assembly that bolts across the bus barassemblies. On bus bar assembly 2730B, these paired connectors arelabeled SFPPPC3B, SFPPPC4B, SFPPPC5B, and SFPPPC6B. In between thepaired connectors, individual power connectors SFPC1B, SFPC3B, SFPC5B,SFPC7B, and SFPC9B serve the remainder of the switch fabric cards.

Particularly with the lower density connector pattern of FIG. 27,additional bus bar features can be incorporated to further reduce noisepropagation through the primary power supply distribution system. Oneexample of additional features is illustrated in FIG. 28, which showsyet another primary power distribution system embodiment 2800 with thesame power connector pattern as embodiment 2700. Embodiment 2800 showstwo bus bar assemblies 2830A and 2830B. Each bus bar assembly, at thepower connector level, contains two power conductors that completelyoverlap except for tabs that extend from each conductor at the locationsof the power connectors. For instance, bus bar assembly 2830B comprisesa supply conductor BS and a return conductor BR that overlap along theirhorizontal run. At the location of power connector PPPC2B, supplyconductor BS contains a tab 2850S projecting upwards, and returnconductor BR contains a tab 2850R projecting downwards. The tabs can bebare copper, or coated with an insulator except for where a powerconnector must make contact with the copper.

An additional feature on the bus bar assemblies 2830A and 2830B is thepresence of ferrite chokes placed between some of the power conversioncard connectors. Bus bar assembly 2830B has two such ferrite chokes,2852 and 2854, placed around BS and BR between the first and secondpower conversion card connectors on each end of the bus bar assembly.

Many options exist for fabrication of bus bar assemblies according to anembodiment. The two conductors of a bus bar assembly can be separated byan “H”-shaped plastic extrusion inserted between the conductors wherethey overlap. The plastic extrusion can snap, screw, clip, etc. to theconductors to form a solid assembly, and can also snap, screw, clip,etc. into the chassis.

In an alternate embodiment, the assembly comprising the two conductorsand the “H”-shaped plastic extrusion (and possibly ferrite chokes,insulated from the conductors), with the contact ends covered, is dippedinto a bath of thermoplastic material. The thermoplastic material, oncecooled and hardened, provides additional insulation for the assembly.Alternately, the two conductors can be held in a desired relativeposition in a fixture during the dipping, cooling, and hardeningoperation, with no plastic spacer at all. A supportive, non-conductivescrim cloth material can also be wrapped around the conductors in eithermethod, thereby adding strength and stiffness to the assembly andhelping to maintain the desired conductor gap. Each conductor may alsobe dipped separately, and then used in any of the above operations withan “H”-shaped plastic extrusion and/or a second dipping operation.

As one alternative to dipping, the pair of conductors can be fixtured atthe correct spacing, with ends covered, and wrapped in a pre-impregnatedthermo-set cloth material, and then cured. As a second alternative todipping, the pair of conductors can be fixtured at the correct spacing,with ends covered, and surrounded with heat shrink tubing lined with ahot-melt adhesive. When the tubing is heated and shrunk, the adhesivelining adheres itself to and fills the gap between the two conductors,forming a flexible insulated assembly. As a third alternative, theconductor pair can be inserted into a mold that protects the ends of theconductors, and then over-molded with a thermoplastic or thermo-setmaterial.

When shielding is placed over the conductor pair, the shielding can beassembled to the conductor pair in several ways. For instance, theshield may consist of: a wire mesh tube that is slid over the insulatedconductor assembly; metallized tape applied over the insulated conductorassembly; a metal coating applied, e.g., by vapor deposition,electro-deposition, or plasma spray; or a metal mesh sheet cut to shapeand wrapped around the insulated bus bar. A drain wire, which mayconsist of a copper braid, is soldered to the shield material. A ringlug is crimped onto the other end of the drain wire, and secured tochassis ground by a backplane mounting screw, preferably placed close tothe backplane end of the bus bar.

Once the shielding is in place, an optional final step can add furtherinsulation over the assembly, including the shielding. Any of theinsulating methods described above can be used to add this furtherinsulation. The drain wire may be insulated as part of this step, or canbe separately insulated if so desired.

Although bus bar features are shown on a bus bar that delivers powerdirectly to the power conversion cards, the disclosed bus bar noiserejection/suppression features can also be used, e.g., on a bus bar fora FIG. 4 type backplane, which bolts to the power delivery end of thebus bar conductors and then internally distributes power to the powerconversion cards. Such a bus bar can alternately be used with a priorart backplane, e.g., like the one shown in FIG. 1, to improve noiserejection on the primary power delivery network of the prior art. Whencooling air does not need to pass behind the backplane, the portion ofthe bus bars extending between the power supply cavity and theelectronics cavity can be shielded from electronic noise by enclosure ina separate cavity within the chassis.

FIGS. 29 and 30 illustrate further features of power flow through alogic power conversion card according to an embodiment. FIG. 29 shows aside view of a logic power conversion card LPCC, including two magnifiedcross-sections showing the use of internal layers of the card's circuitboard. LPCC receives primary power through two connector blocks PPCBAand PPCBB. The two connector blocks feed power to internal conductivelayers of the LPCC circuit board. Cross-section 2910 shows one possiblelayer assignment near the primary power connector blocks. All labeledlayers are conductive layers constructed of one-ounce copper, separatedby FR-4 dielectric layers.

Cross-section 2910 depicts 12 conductive layers. The two outermostconductive layers are DGND digital ground layers (digital groundconnections to card LPCC are made, e.g., through two logic powerconnectors LPCBA and LPCBB). Each DGND layer provides radiated noiseshielding for the interior conductive layers of the circuit board. EachDGND layer is separated by a dielectric layer from a conductivesignaling layer (SIG A, bottom, and SIG B, top). The signaling layersare used to route control and monitoring signal traces on the LPCCcircuit board, e.g., from logic power connectors LPCBA and LPCBB. Asshown, the signaling layers can be placed much closer to the outer DGNDlayers than to the next innermost conductive layers, such that thesignaling impedance is dominated by the DGND proximity, and returnsignaling currents flow primarily in the outer DGND layers.

Below the SIG B layer, the next four insulated conductive layers carry48V B RTN, 48V B supply, 48V B RTN, and 48V B supply, respectively.Likewise, above the SIG A layer, the next four insulated conductivelayers carry 48V A RTN, 48V A supply, 48V A RTN, and 48V A supply,respectively. The innermost 48V A and 48V B supply layers face eachother, separated by a dielectric layer. Capacitance between thereturn-supply-return-supply layer stacks is enhanced, providing furthernoise attenuation in the primary power supply circuits.

Primary power passes from connectors PPCBA and PPCBB to isolation andfiltering block IFB (described further below in conjunction with FIG.30), where power from the A and B supplies is combined, furtherfiltered, and distributed to logic power supply conversion circuits (PS1and PS2 are visible in FIG. 29).

At the output side of the logic power conversion circuits, theconductive layers that were used on the primary power supply side of thelogic power conversion card to carry primary power are now retasked tocarry logic power to the logic power connectors LPCBA and LPCBB. On thelogic power delivery side of the logic power conversion card, forinstance, the 12 conductive layers of the LPCC circuit board can beassigned, top to bottom as shown in cross-section 2920, as DGND, SIG B,DGND, LV1, DGND, LV2, LV3, DGND, LV4, DGND, SIG A, and DGND, where “LV”denotes a logic voltage. Depending on the current needs of each logicvoltage, more conductive layers or less of a conductive layer can beassigned to each logic voltage. A layer may also be split, if isolationcircuitry SBR is included on the card and the same power supplyconversion circuit supplies power to two or more logic cards.

FIG. 30 contains a block diagram for the power conversion circuitry 3000on a logic power conversion card according to an embodiment. Circuitry3000 includes isolation and filtering block IFB, four logic powersupplies PS1 to PS4, and split bridge SBR.

Isolation and filtering block IFB contains a diode bridge DB, adifferential/common mode filter DCM, and four common mode filters CM1 toCM4. Diode bridge DB allows the 48V “A” and “B” supplies to be connectedtogether at the input of filter DCM, while preventing loops between theprimary supplies as a result of the common connection. Filter DCM works,in conjunction with the capacitance of the primary power planes in thecard itself, to reduce noise in the combined primary power supply. Eachof common mode filters CM1 to CM4 connects the output of filter DCM to arespective input of one of power supplies PS1 to PS4. The common modefilters reduce noise from each power supply feeding into the inputs ofthe other power supplies.

Each power supply PS1 to PS4 is set to output a corresponding logicvoltage LV1 to LV4. In other embodiments, two or more power supplies canteam to produce current at the same logic voltage, or a different numberof logic voltages can be generated. The “CTL” input to each power supplycan comprise voltage feedback signals, or instructions from the chassiscontroller, from the supplied line cards, or from an onboard controllerlocated on the power conversion card.

When the power conversion card is designed to supply power to multipleline cards, a split bridge SBR can provide isolation and/or common modefiltering for each supplied voltage, as desired. In FIG. 30, splitbridge SBR is shown as a simple diode inserted in each leg of a splitoutput for each power supply. The split bridge can optionally employfurther noise filters, or even controlled solid-state switches todynamically route logic power and/or combine logic power from differentpower supplies.

Those skilled in the art will appreciate that the embodiments and/orvarious features of the embodiments can be combined in ways other thanthose described. The specific embodiments can be adapted to otherbackplane chassis arrangements with different logic card numbers and/orfunctionality, positive pressure cooling, side-to-side cooling, etc.Cooling air for logic cards and power converter cards need not enterfrom the same cooling air intake, exit through the same exhaust, or usecommon air movers. When different backplane circuit boards are used forsignaling and for primary power and/or logic power delivery,noise-defeating structures and shields can be employed between thecircuit boards. Noise shielding can also be employed between thesignaling portion of a backplane and the power conversion cards runningbehind the backplane as well. Not all logic voltages supplied by a powerconversion card need be supplied to the same group of logic cards, wherea power conversion card provides power to two or more logic cards.

Although the specification may refer to “an”, “one”, “another”, or“some” embodiment(s) in several locations, this does not necessarilymean that each such reference is to the same embodiment(s), or that thefeature only applies to a single embodiment.

1. A modular packet network device comprising: a chassis; a plurality oflogic cards located in a first cooling path within the chassis; anelectrical signaling backplane having front side signaling connectors towhich the plurality of logic cards connect, the electrical signalingbackplane distributing signaling between the plurality of logic cards; aplurality of power converter cards located in a second cooling pathwithin the chassis, the second cooling path passing substantially behindthe backplane, the plurality of power converter cards converting primarypower to logic power, each power converter card supplying logic power toa strict subset of the plurality of logic cards; a primary powerdistribution backplane having primary power connectors, the primarypower distribution backplane distributing primary power to the pluralityof power converter cards through the primary power connectors; and atleast two redundant primary power supplies, the primary powerdistribution backplane independently distributing power from at leasttwo of the redundant primary power supplies to each of the powerconverter cards through the primary power connectors.
 2. The modularpacket network device of claim 1, wherein the electrical signalingbackplane and primary power distribution backplane are arrangedvertically in the chassis, the primary power distribution backplaneextending below the electrical signaling backplane.
 3. The modularpacket network device of claim 2, wherein the primary power distributionbackplane lies behind the plane of the electrical signaling backplane.4. The modular packet network device of claim 1, further comprising alogic power distribution backplane having logic power connectors, thelogic power distribution backplane distributing logic power from theplurality of power converter cards to the plurality of logic cardsthrough the logic power connectors.
 5. The modular packet network deviceof claim 4, wherein the logic power distribution backplane is arrangedvertically in the chassis, the logic power distribution backplaneextending above the electrical signaling backplane.
 6. The modularpacket network device of claim 5, wherein the logic power distributionbackplane lies behind the plane of the electrical signaling backplane.